FIFTH system for general-purpose connectionist computation

To date, work on formalizing connectionist computation in a way that is at least Turingcomplete has focused on recurrent architectures and developed equivalences to Turing machines or similar super-Turing models, which are of more theoretical than practical significance. We instead develop connectionist computation within the framework of information propagation networks extended with unbounded recursion, which is related to constraint logic programming and is more declarative than the semantics typically used in practical programming, but is still formally known to be Turing-complete. This approach yields contributions to the theory and practice of both connectionist computation and programming languages. Connectionist computations are carried out in a way that lets them communicate with, and be understood and interrogated directly in terms of the high-level semantics of a general-purpose programming language. Meanwhile, difficult (unbounded-dimension, NP-hard) search problems in programming that have previously been left to the programmer to solve in a heuristic, domain-specific way are solved uniformly a priori in a way that approximately achieves information-theoretic limits on performance. Connectionist augmentations for representing high-dimensional spaces concisely with information-theoretic autoencoders are juxtaposed with program structures along the boundaries of program recursion, then composed in a hierarchy along branches of the recursion. The compression that results permits communicating information in arbitrarily deep recursions at costs proportional to the logarithm of the recursion depth and information to be communicated, and permits identification of similar subtrees of the recursion. This forms the basis of a general and efficient means of solving search, planning, scheduling, and optimization problems guided by learned connectionist representations of the search space. It also rediscovers the essence of biologically-inspired hierarchical models of information processing in cortex, establishing the relevance of the same concepts to theory of computation in terms of information-theoretic approximation. Results generalize work on strictly linear and harmonic state-space compression in POMDPs.